Skip to content

Conversation

@davidli1997
Copy link
Contributor

@davidli1997 davidli1997 commented Oct 30, 2025

Changelist

Bug Fix: /transfers/parentSubaccountNumber returning only ~10 results instead of respecting limit

Problem:
The /transfers/parentSubaccountNumber endpoint was incorrectly returning only 10-11 transfers even when no limit was specified (default should be 1000).

Root Cause:
The endpoint had a two-step process:

  1. Query database for up to 1000 transfers involving all child subaccounts of a parent
  2. Filter out same-parent transfers (e.g., transfers between subaccount 0 and subaccount 128, both parent 0) in memory

When a parent subaccount had many internal transfers between its children, the database would return 1000 transfers (mostly same-parent), filter them out, and leave only ~10 cross-parent transfers.

Solution:

  • Created new database function findAllToOrFromParentSubaccount() that filters same-parent transfers at the SQL level
  • Uses LEFT JOINs with the subaccounts table to filter transfers where both sender and recipient have the same address AND same parent subaccount number (subaccountNumber % 128)
  • The SQL filter runs BEFORE the LIMIT is applied, ensuring the correct number of cross-parent transfers are returned

Changes:

  1. New Database Function (packages/postgres/src/stores/transfer-table.ts)

    • Added findAllToOrFromParentSubaccount() with SQL-level filtering
    • Added ParentSubaccountTransferQueryConfig interface
    • Filters same-parent transfers using: (sender_sa.subaccountNumber % 128) = (recipient_sa.subaccountNumber % 128)
  2. Updated Controller (indexer/services/comlink/src/controllers/api/v4/transfers-controller.ts)

    • Modified getTransfersForParentSubaccount() to use new database function
    • Removed in-memory filtering logic (now handled in SQL)

Test Plan

Key Test Cases

  1. Same-parent filtering: Verified transfers between child subaccounts of same parent (e.g., 0→128) are excluded
  2. Cross-parent transfers: Verified transfers between different parents (e.g., parent 0→parent 1) are included
  3. Deposits/Withdrawals: Verified wallet↔subaccount transfers are included
  4. Limit validation: Created 35 same-parent + 15 cross-parent transfers, verified all 15 cross-parent transfers returned (not just 10)
  5. Pagination: Verified pagination works correctly with SQL-level filtering

Manul Testing on Staging

curl "https://indexer.v4staging.dydx.exchange/v4/transfers/parentSubaccountNumber?address=dydx1nzuttarf5k2j0nug5yzhr6p74t9avehn9hlh8m&parentSubaccountNumber=0" | jq '{transferCount: (.transfers | length), totalResults: .totalResults, pageSize: .pageSize, offset: .offset}'

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  427k  100  427k    0     0   278k      0  0:00:01  0:00:01 --:--:--  278k
{
  "transferCount": 1000,
  "totalResults": null,
  "pageSize": null,
  "offset": null
}

Now it retrieves up the API limit - 1,000

Author/Reviewer Checklist

  • If this PR has changes that result in a different app state given the same prior state and transaction list, manually add the state-breaking label.
  • If the PR has breaking postgres changes to the indexer add the indexer-postgres-breaking label.
  • If this PR isn't state-breaking but has changes that modify behavior in PrepareProposal or ProcessProposal, manually add the label proposal-breaking.
  • If this PR is one of many that implement a specific feature, manually label them all feature:[feature-name].
  • If you wish to for mergify-bot to automatically create a PR to backport your change to a release branch, manually add the label backport/[branch-name].
  • Manually add any of the following labels: refactor, chore, bug.

Summary by CodeRabbit

  • New Features

    • Transfer query updated to exclude transfers between child subaccounts of the same parent and to support pagination, height/time filters, and deposits/withdrawals.
  • Behavior Changes

    • API now returns the raw transformed transfer results from the parent-aware query (client-side parent filtering removed).
  • Tests

    • Comprehensive tests added/updated to cover cross-parent vs same-parent transfers, pagination, limits, and edge cases.

@davidli1997 davidli1997 requested a review from a team as a code owner October 30, 2025 15:40
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 30, 2025

Walkthrough

A new DB query function, findAllToOrFromParentSubaccount, fetches transfers to/from child subaccounts of a parent while excluding transfers between same-parent children. The transfers controller now calls this DB function and tests were added/updated to cover filtering, pagination, and time/height constraints.

Changes

Cohort / File(s) Summary
Type Definitions
indexer/packages/postgres/src/types/query-types.ts
Added ParentSubaccountTransferQueryConfig interface (subaccount IDs, limit, page, createdBeforeOrAt, createdBeforeOrAtHeight).
Transfer Table Implementation
indexer/packages/postgres/src/stores/transfer-table.ts
Added exported findAllToOrFromParentSubaccount that builds a query joining subaccounts and applies a NOT(...) exclusion to omit same-parent child-to-child transfers; supports height/time filters, ordering, and pagination.
Transfer Table Tests
indexer/packages/postgres/__tests__/stores/transfer-table.test.ts
New comprehensive tests for findAllToOrFromParentSubaccount covering same-parent exclusion, deposits/withdrawals, cross-parent transfers, limits, pagination, height/time filters, and empty results.
Transfers Controller
indexer/services/comlink/src/controllers/api/v4/transfers-controller.ts
Replaced findAllToOrFromSubaccountId call with findAllToOrFromParentSubaccount; removed client-side parent-subaccount filtering.
Controller Tests
indexer/services/comlink/__tests__/controllers/api/v4/transfers-controller.test.ts
Updated tests to seed blocks/events and assert parent-subaccount filtering via the new DB query; imports adjusted and assertions simplified with arrayContaining/objectContaining.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant Controller as Transfers Controller
    participant DB as Transfer Table
    participant Subaccount as Subaccount Table

    Client->>Controller: GET /transfers?parentSubaccountNumber=X
    Controller->>DB: findAllToOrFromParentSubaccount({ subaccountId, limit, page, ... })

    rect rgb(235, 245, 255)
    note right of DB: Server-side parent-subaccount flow
    DB->>Subaccount: Join to map parent -> child subaccount IDs
    Subaccount-->>DB: child subaccount IDs
    DB->>DB: WHERE (to_id IN child_ids OR from_id IN child_ids)\nAND NOT (from_id IN child_ids AND to_id IN child_ids)
    DB->>DB: Apply createdBeforeOrAt / createdBeforeOrAtHeight, order, limit/page
    end

    DB-->>Controller: Filtered transfers (cross-parent only)
    Controller-->>Client: JSON response
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

  • Pay attention to the NOT(...) join exclusion logic in transfer-table.ts for correctness.
  • Verify pagination/total-count queries and edge cases.
  • Review new tests for completeness and potential flakiness (time/height seeding).

Possibly related PRs

Suggested labels

indexer-postgres-improvement

Suggested reviewers

  • tqin7
  • shrenujb

Poem

🐰 I hopped through rows and joined with care,

To find child hops and filter pairs.
Sibling hops are gently left behind,
Cross-parent paths are what I find.
A tiny carrot for queries well-defined.

Pre-merge checks and finishing touches

✅ Passed checks (2 passed)
Check name Status Explanation
Description check ✅ Passed The description includes all required sections: detailed Changelist explaining the problem, root cause, and solution; comprehensive Test Plan with key test cases and manual staging validation; and complete Author/Reviewer Checklist with all applicable items present.
Title check ✅ Passed The title accurately describes the main change: moving parent subaccount transfer filtering from client-side to SQL to respect the limit parameter, which is the core bug fix in this PR.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch davidli/transfers_endpoint

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 55bdb1a and f01efaa.

📒 Files selected for processing (4)
  • indexer/packages/postgres/__tests__/stores/transfer-table.test.ts (3 hunks)
  • indexer/packages/postgres/src/stores/transfer-table.ts (2 hunks)
  • indexer/packages/postgres/src/types/query-types.ts (2 hunks)
  • indexer/services/comlink/src/controllers/api/v4/transfers-controller.ts (5 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: hwray
PR: dydxprotocol/v4-chain#2551
File: protocol/x/subaccounts/keeper/subaccount.go:852-865
Timestamp: 2024-11-15T16:00:11.304Z
Learning: The function `GetCrossInsuranceFundBalance` in `protocol/x/subaccounts/keeper/subaccount.go` already existed and was just moved in this PR; changes to its error handling may be out of scope.
Learnt from: hwray
PR: dydxprotocol/v4-chain#2551
File: protocol/x/subaccounts/keeper/subaccount.go:833-850
Timestamp: 2024-11-15T15:59:28.095Z
Learning: The function `GetInsuranceFundBalance` in `protocol/x/subaccounts/keeper/subaccount.go` already existed and was just moved in this PR; changes to its error handling may be out of scope.
📚 Learning: 2024-11-15T16:00:11.304Z
Learnt from: hwray
PR: dydxprotocol/v4-chain#2551
File: protocol/x/subaccounts/keeper/subaccount.go:852-865
Timestamp: 2024-11-15T16:00:11.304Z
Learning: The function `GetCrossInsuranceFundBalance` in `protocol/x/subaccounts/keeper/subaccount.go` already existed and was just moved in this PR; changes to its error handling may be out of scope.

Applied to files:

  • indexer/packages/postgres/src/stores/transfer-table.ts
🧬 Code graph analysis (3)
indexer/services/comlink/src/controllers/api/v4/transfers-controller.ts (3)
indexer/packages/redis/__tests__/caches/constants.ts (1)
  • address (15-15)
indexer/services/comlink/src/lib/errors.ts (1)
  • NotFoundError (15-20)
indexer/services/comlink/src/types.ts (1)
  • ParentSubaccountTransferResponseObject (223-245)
indexer/packages/postgres/__tests__/stores/transfer-table.test.ts (3)
indexer/packages/postgres/__tests__/helpers/mock-generators.ts (1)
  • seedData (58-107)
indexer/packages/postgres/src/helpers/db-helpers.ts (3)
  • migrate (176-178)
  • clearData (143-159)
  • teardown (180-186)
indexer/packages/postgres/__tests__/helpers/constants.ts (10)
  • defaultSubaccountId (156-159)
  • isolatedSubaccountId (176-179)
  • defaultAsset (217-222)
  • defaultTendermintEventId (495-499)
  • createdDateTime (70-70)
  • createdHeight (71-71)
  • defaultSubaccountId2 (160-163)
  • defaultTendermintEventId2 (500-504)
  • defaultAddress (74-74)
  • defaultWalletAddress (83-83)
indexer/packages/postgres/src/stores/transfer-table.ts (7)
indexer/packages/postgres/src/types/query-types.ts (2)
  • ParentSubaccountTransferQueryConfig (258-266)
  • QueryConfig (111-114)
indexer/packages/postgres/src/types/utility-types.ts (1)
  • Options (15-25)
indexer/packages/postgres/src/constants.ts (1)
  • DEFAULT_POSTGRES_OPTIONS (124-127)
indexer/packages/postgres/src/types/pagination-types.ts (1)
  • PaginationFromDatabase (1-6)
indexer/packages/postgres/src/types/db-model-types.ts (1)
  • TransferFromDatabase (167-178)
indexer/packages/postgres/src/helpers/stores-helpers.ts (2)
  • verifyAllRequiredFields (14-24)
  • setupBaseQuery (47-60)
indexer/packages/postgres/src/models/transfer-model.ts (1)
  • TransferModel (8-150)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (28)
  • GitHub Check: (Public Testnet) Build and Push ECS Services / call-build-and-push-bazooka-lambda / (bazooka) Build and Push Lambda
  • GitHub Check: (Public Testnet) Build and Push ECS Services / call-build-and-push-vulcan / (vulcan) Build and Push
  • GitHub Check: (Public Testnet) Build and Push ECS Services / call-build-and-push-ecs-service-roundtable / (roundtable) Build and Push
  • GitHub Check: (Public Testnet) Build and Push ECS Services / call-build-and-push-ecs-service-socks / (socks) Build and Push
  • GitHub Check: (Public Testnet) Build and Push ECS Services / call-build-and-push-auxo-lambda / (auxo) Build and Push Lambda
  • GitHub Check: (Public Testnet) Build and Push ECS Services / call-build-and-push-ecs-service-ender / (ender) Build and Push
  • GitHub Check: (Mainnet) Build and Push ECS Services / call-build-and-push-auxo-lambda / (auxo) Build and Push Lambda
  • GitHub Check: (Mainnet) Build and Push ECS Services / call-build-and-push-bazooka-lambda / (bazooka) Build and Push Lambda
  • GitHub Check: (Public Testnet) Build and Push ECS Services / call-build-and-push-ecs-service-comlink / (comlink) Build and Push
  • GitHub Check: (Mainnet) Build and Push ECS Services / call-build-and-push-vulcan / (vulcan) Build and Push
  • GitHub Check: (Mainnet) Build and Push ECS Services / call-build-and-push-ecs-service-comlink / (comlink) Build and Push
  • GitHub Check: (Mainnet) Build and Push ECS Services / call-build-and-push-ecs-service-roundtable / (roundtable) Build and Push
  • GitHub Check: call-build-ecs-service-vulcan / (vulcan) Check docker image build
  • GitHub Check: (Mainnet) Build and Push ECS Services / call-build-and-push-ecs-service-ender / (ender) Build and Push
  • GitHub Check: call-build-ecs-service-ender / (ender) Check docker image build
  • GitHub Check: call-build-ecs-service-roundtable / (roundtable) Check docker image build
  • GitHub Check: call-build-ecs-service-socks / (socks) Check docker image build
  • GitHub Check: call-build-ecs-service-comlink / (comlink) Check docker image build
  • GitHub Check: (Mainnet) Build and Push ECS Services / call-build-and-push-ecs-service-socks / (socks) Build and Push
  • GitHub Check: check-build-auxo
  • GitHub Check: check-build-bazooka
  • GitHub Check: test / run_command
  • GitHub Check: build-and-push-testnet
  • GitHub Check: build-and-push-mainnet
  • GitHub Check: run_command
  • GitHub Check: Analyze (javascript-typescript)
  • GitHub Check: Analyze (go)
  • GitHub Check: Summary

@davidli1997 davidli1997 changed the title Fix: Move parent subaccount transfer filtering to SQL to respect limit parameter [ENG-1188] Move parent subaccount transfer filtering to SQL to respect limit parameter Nov 5, 2025
@linear
Copy link

linear bot commented Nov 5, 2025

): Promise<PaginationFromDatabase<TransferFromDatabase>> {
verifyAllRequiredFields(
{
[QueryableField.LIMIT]: limit,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we set an upper limit for this? and validate that limit is greater than 0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are already doing this on the API level
{"errors":[{"value":-1,"msg":"limit must be a positive integer that is not greater than max: 1000","param":"limit","location":"query"}]}
{"errors":[{"value":0.5,"msg":"limit must be a positive integer that is not greater than max: 1000","param":"limit","location":"query"}]}
{"errors":[{"value":1001,"msg":"limit must be a positive integer that is not greater than max: 1000","param":"limit","location":"query"}]}

@jusbar23 jusbar23 self-requested a review November 10, 2025 20:26
@davidli1997 davidli1997 merged commit 9be7ed2 into main Nov 10, 2025
33 checks passed
@davidli1997 davidli1997 deleted the davidli/transfers_endpoint branch November 10, 2025 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

3 participants